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LONG  TERM  GOALS 

The  goal  of  this  project  was  to  procure  computing  equipment  configured  for  our  ocean  acoustics 
research.  This  work  benefits  from  a  strong  computational  capability.  Limits  of  computational 
accuracy  can  be  tested  against  theory,  practicality  of  computational  methods  can  be  determined,  and 
studies  of  underwater  acoustics  phenomena  can  be  undertaken  with  computational  methods. 

OBJECTIVES 

The  three-dimensional  acoustic  propagation  models  that  we  depend  on  at  this  time  are  not  written  in 
codes  that  make  them  easily  parallcli zablc  in  the  manner  that,  for  example,  atmospheric  or  ocean 
general  circulation  models  (GCMs)  are  parallel.  Many  GCMs  can  split  a  three-dimensional 
computational  domain  in  to  subdomains  and  pass  the  boundary  conditions  back  and  forth,  and  each 
subdomain  can  be  run  independently  on  processing  units  with  access  to  a  typically  available  amount  of 
memory,  say  16  or  32  gigabytes.  Our  models  require  each  processor  to  have  access  to  up  to  500 
gigabytes  of  memory  in  their  current  configurations,  although  meaningful  results  can  be  obtained  with 
64  gigabytes  if  the  frequency  is  kept  below  1000  Hz  and  the  water  depth  is  less  than  1 500  m. 

As  an  alternative  to  completely  reformulating  our  computational  methods  to  allow  results  to  be 
obtained  with  limited  amounts  of  memory  available  to  individual  processing  units  (with  no 
time  frame  for  successful  completion  known  at  this  time),  we  sought  to  obtain  powerful 
computational  engines,  which  are  routinely  available,  that  are  compatible  with  the  architectures  of  our 
models. 

APPROACH 

As  outlined  above,  each  processor  in  the  acoustic  calculation  must  access  a  large  amount  of  memory  at 
this  time.  This  is  incompatible  with  most  current-generation  High-Performance  Computing  Clusters, 
which  have  thousands  of  linked  processors,  but  with  the  clusters  grouped  into  nodes  of  16  or  32  each 
with  direct  access  only  to  memory  allocated  to  that  node.  Many  computational  toots  arc  written  so  that 
the  problem  can  be  divided  into  segments  solvable  on  many  processor  units  each  needing  only  a  subset 
of  the  total  required  memory.  This  is  not  true  for  current  generation  3D  acoustic  parabolic  equation 
solvers. 
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Our  straightforward  approach  to  increase  our  computing  capacity  was  to  purchase  specialized 
computer  workstations  with  a  large  amount  of  memory  accessible  by  ail  of  the  processors.  Wc  have 
purchased  three  workstations,  each  with  many  CPU  cores  to  enable  parallel  computing,  with  all 
memory  space  available  to  every  core. 

WORK  COMPLETED 

Three  desktop  workstations  have  been  procured,  tested,  and  put  into  use.  One  file  server  computer  to 
store  simulation  output  has  also  been  purchased.  The  first  workstation  has  28  CPU  cores,  dual  “thread, 
(56  processors,  effectively),  and  512  GB  memory.  The  second  has  24  CPU  cores,  dual-thread,  (48 
processors,  effectively),  and  512  GB  memory.  The  third  has  28  CPU  cores,  dual-thread,  (56 
processors,  effectively),  and  256  GB  memory.  Mr.  Arthur  Newhall  of  WHOl  worked  with  the  vendors 
to  secure  the  best  prices  for  workstations  of  the  highest  capability. 

Two  figures  show  aspects  of  a  test  of  Workstation  I .  lQQO-Hz  sound  is  simulated  to  propagate 
eastward  up  a  slope  and  into  a  canyon  from  San  Diego  Trough  (Figure  1 ),  The  split-step  Fourier  code 
described  in  Lin  ct  al.  (2013)  is  used,  with  computational  grid  of  32,768  by  32,768  implements  for 
each  marching  step.  The  propagation  distance  is  13.6  km  and  the  domain  width  is  over  8  km.  Compute 
wall-clock  time  is  21 0  hours,  with  14  threads  used  for  acoustic  medium  interpolation  and  28  threads 
used  for  propagation  (can  possibly  run  faster).  This  uses  a  maximum  of -250  gigabytes  memory. 
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Figure  1 .  The  test  domain  is  shown.  San  Diego  Bay  is  at  the  top  right.  The  bathymetry  is  shown  with 
100-m  filled  contours.  The  box  shows  the  x/y  limits  of  the  computational  domain  with  the  source 
shown  with  a  circle  along  the  x=0  side  and  the  arrow  pointing  in  the  positive  x  (propagation)  direction. 
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Figure  2.  The  magnitude  and  phase  of  the  computed  field  at  range  x=180  m  (lose  to  the  source)  are 
shown  at  the  top  and  bottom,  respectively,  Source  level  is  0  dB.  The  point  source  is  at  25  m  depth. 

Purchased  Item  Specifications: 

Workstation  1:  From  Pogo  Linux,  Inc, 

Velocity  D58SQ: 

Whisper  Quiet  Workstation  (28dB)  with  High-end  Graphics  Support 

-  Dual  Intel  Xeon  E5-2600v3  series  with  QPI  up  to  9.6GT/s 

-  Up  to  1TB  DDR4  ECC  Registered  DIMM 

-  Intel  Dual  i210  Gigabit  Ethernet  Controller 

-  8  x  3.5"  hot-swap  SAS/  SATA  Drive  Bays 

-  1200W  Low-Noise  High  Efficiency  Power  Supply 

2  Intel  Xeon  E5-2695v3  2.3GHz  35MB  Cache  9.6GT/s  (iota!  cores  28,  threads  56) 

I  512GB  DDR4  2!33MHz  ECC  Reg  (16  x  32GB) 

1  Intel  S35 10  Scries  1.6TB  MLC  SSD  6Gb/s 
1  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 

1  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 

I  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 

I  Seagate  Constellation  ESA  2TB  7200RPM  128MB  SATA  3.0  HDD 

1  LSI  MegaRAlD  SAS  9361-4i  12Gb/s  SAS  Controller 
I  NVIDIA  Quadro  K2200  4GB  PCI-E  lx  DVI/2x  DisplayPort 
I  Intel  10G  Dual  Port  RJ45  Ethernet  Adapter 
1  SATA  24xDVD-RW  5.25" 

1  No  Operating  System.  Include  testing  and  customer  OS  preference  in  notes. 

1  Custom  RAID  Configuration  -  Add  instructions  to  system  notes 
i  Logitech  Desktop  MK.120 
1  Creative  Inspire  P7800  7. 1  Speaker  System 
!  Samsung  SyncMaster  24IN  WS  LCD  1920X1200 

1  Return  to  Depot  Warranty  (3  Year  Hardware  Warranty  with  Standard  Advance  Parts  Replacement) 
Notes:  Install  and  test  with  latest  stable  version  of  Linux  Mint 


at  x-160 m. 
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Data-storage  unit  (file  server):  From  Thinkmate 

STX-JB  JE16-0310-SAS3 

Power  Estimate:  Watts:  158.4,  Volt-Amps:  162.36,  BTU/h:  540.46, 

Amps(llOV):  1 .44,  Amps(208V):  0.76 

-  Supcrmicro  SupcrChassis  836BE1C-R1K03JBOD  -  3U  -  16  x  3.5"  SAS3  -  I2Gb/s  Single  Expander  - 
1 000W  Redundant 

-  16  x  8.0TB  SATA  6.0Gb/s  7200RPM  -  3.5"  -  Hitachi  Ultrastar  IIe8  (512e) 

-  Use  an  existing  Host  Server  or  Adapter 

-  3  Year  Advanced  Parts  Replacement  Warranty 

Workstation  2:  From  Pogo  Linux,  Inc: 

Velocity  D58SQ 

Whisper  Quiet  Workstation  (28dB)  with  High-end  Graphics  Support 

-  Dual  Intel  Xeon  E5-2600  v4  series  with  QP1  up  to  9.6GT/s 

-  Up  to  1TB  DDR4  ECC  Registered  DIMM 

-  Intel  Dual  i21 0  Gigabit  Ethernet  Controller 

-  8  x  3.5"  hot-swap  SAS/  SATA  Drive  Bays 

-  I200W  Low-Noise  High  Efficiency  Power  Supply 

2  Intel  Xeon  E5-2687Wv4  12C  3.0GHz  30MB  Cache  (total  cores  24.  threads  48) 

1  512GB  DDR4  2133MHz  ECC  Reg  (16  x  32GB) 

1  Intel  S3610  Series  1.2TB  MLC  SSD  6Gb/s 

1  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 
1  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 
1  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 
1  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 
I  NVIDIA  Quadro  K2200  4GB  PCI-E  lx  DVI/2x  DisplayPort 
1  Intel  10G  Dual  Port  RJ45  Ethernet  Adapter 
1  SATA  24x  DVD-RW  5.25" 

1  No  Operating  System.  Include  testing  and  customer  OS  preference  in  notes, 

1  Custom  RAID  Configuration  -  Add  instructions  to  system  notes 
1  Logitech  Desktop  MK 1 20 
1  Samsung  SyncMaster  24IN  WS  LCD  1920X1200 

1  Return  to  Depot  Warranty  (3  Year  Hardware  Warranty  with  Standard  Advance  Parts  Replacement) 
Notes:  Install  and  test  with  latest  stable  version  of  Linux  Mint 
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Workstation  3:  From  Pogo  Linux,  Inc: 

Velocity  D58SQ 

Whisper  Quiet  Workstation  (28dB)  with  High-end  Graphics  Support 

-  Dual  Intel  Xcon  E5-2600  v4  series  with  QPI  up  to  9.6GT/s 

-  Up  to  ITB  DDR4  ECC  Registered  DIMM 

-  Intel  Dual  i2 1 0  Gigabit  Ethernet  Controller 

-  8  x  3.5"  hot-swap  SAS/  SATA  Drive  Bays 

-  1200W  Low-Noise  High  Efficiency  Power  Supply 

2  Intel  Xcon  E5-2680v4  14C  2.4GIIz  35MB  Cache  (total  cores  28.  threads  56) 

1  256GB  DDR4  2 1 33MHz  ECC  Reg  (8  x  32GB) 

1  Intel  S3610  Series  800GB  MLC  SSD  6Gb/s 
1  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 

I  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 

1  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 

1  Seagate  Constellation  ES.3  2TB  7200RPM  128MB  SATA  3.0  HDD 

1  NVIDIA  Quadro  K2200  4GB  PCI-E  lx  DVI/2x  DisplayPort 
1  Intel  I0G  Dual  Port  RJ45  Ethernet  Adapter 
I  SATA  24x  DVD-RW  5.25" 

1  No  Operating  System.  Include  testing  and  customer  OS  preference  in  notes. 

1  Custom  RAID  Configuration  -  Add  instructions  to  system  notes 
1  Logitech  Desktop  MK120 
1  Samsung  SyncMaster  24IN  WS  LCD  1920X1200 

1  Return  to  Depot  Warranty  (3  Year  Hardware  Warranty  with  Standard  Advance  Parts  Replacement) 
Notes:  Install  and  test  with  latest  stable  version  of  Linux  Mint 
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